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DETAILED ACTION 



At the outset, it is noted that U.S. Patent No. 5,754,760 has been issued for U.S. Patent 
Application No. 08/655,149, which is incorporated to the present application. Subsequently, the 
present application should be amended to reflect this. 



The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

Claimsl through 3, 6, and 1 1 through 15 are rejected under 35 U.S.C. 102(b) as being 
clearly anticipated by European Patent Application number 869433 attributed to Anodide et al. 

Anodide et al. discloses a methodology for constructing a graphical representation of a 
GUI comprised of the following actions: 

1 . Starting the GUI software 

2. A "tour of the interface" is performed by successively invoking each window of the 
GUI. Anodide et al. states that this "tour" may be performed manually or instead, be automated 
(see column 12, line 1). 

3. A "GUI object forest" and an "Opens Relation" are constructed (see column 8, Hne 
52). The GUI object forest is a collection of trees; each tree representing a window that appears 



Claim Rejections - 35 USC §102 
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in the GUI and that is not contained in any other window. The roots of these trees represent the 
window itself, while the nodes at level i+1 represent GUI objects that are contained in an object 
at level i. The "Opens Relation" is a set of (op, win) pairs; where performing the operation, op, 
causes the window, win, to open. It is assumed that the "GUI object forest" and the "Opens 
Relation" can be constructed as the "tour of the interface" is performed. 

4. From the "GUI object forest" and "Opens Relation", a graph of the GUI is created. 
Each node in the graph corresponds to a root of a tree in the "GUI object forest" and the edges of 
the graph are inserted according to the "Opens Relation". Inherently, this means that the nodes 
in the graph correspond to windows in the GUI and that edges in the graph correspond to 
operations that open the windows they point to. 

Referring to claim 1, the creation of trees in the "GUI object forest" and (op, win) pairs in 
the "Opens Relation" for the active windows in the interface, as described in the methodology of 
Anodide et al. above, is equivalent to identifying a first set of windows that are active on the 
desktop of the computer. 

Performing an action on a graphical user interface object in a window of the application 
is inherent in performing a "tour of the interface." As explained above, and in Anodide et al., the 
"tour of the interface" requires opening each window in the GUI. It is a common practice in the 
art to allow GUI windows to open via GUI objects in active windows on the desktop of a 
computer. Thus, by performing actions on GUI objects in the windows of the application, a 
"tour of the interface" can be carried out. 
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The "GUI object forest" and "Opens Relation", described by Anodide et al., anticipates 
identifying a second set of windows that are active on the desktop of the computer. The 
disclosure of Anodide et al. implies that for a second set of windows on the desktop, a (op, win) 
pair in the "Opens Relation" is created or already exists for each active window. Similarly for 
each active window in the second set, a tree is created or already exists in the "GUI object 
forest" whose nodes represent GUI objects in the window. An ordering of the pairs in the 
relation and of the nodes in the trees, as is common in the art, can be considered equivalent to 
enumerating the windows and GUI objects of the second set of windows. 

Furthermore, it is impHed by Anodide et al. that the first set of windows must be 
compared to the second set of windows to identify a new window in the second set. Anodide et 
al. acknowledges that a given window in the application could be opened by various different 
operations and that a given operation could open several different windows. Consequently, the 
operation-window pair of the "Opens Relation" of Anodide et al. has a many-to-many 
relationship (column 9, line 8). Therefore it can be deducted that as "Opens Relation" (op, win) 
pairs are created for a second set of active windows on the desktop, it must be determined if each 
window in the second set already exists in the relation. Thus, each window in the second set is 
compared to windows that are already in the "Opens Relation", which includes windows in the 
first set. 

Finally, creating a graph from the "Opens Relation" and the "GUI object forest", as 
explained by Anodide et al., is equivalent to adding a new window to a map of the graphical user 
interface of the application. As Anodide et al. discloses, a new window that is encountered 
during the "tour of the interface", has a tree created for it in the "GUI object forest" and a (op, 
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win) pair in the in the "Opens Relation", from this, a node is created for the tree in the graph 
representing the GUI of the application. 

Referring to claim 2, Anodide et al. discloses an "Opens Relation", which, as described 
above, each operation-window pair of the "Opens Relation" has a many-to-many relationship 
(column 9, line 8). Therefore it can be deducted that as "Opens Relation" (op, win) pairs are 
created for a set of active windows on the desktop, which could include new windows, it must be 
determined if each window in the set already exists in the relation. Thus, each window in the set 
is compared to windows that are already in the "Opens Relation". Since the "Opens Relation" is 
used to construct the graph of the GUI, this is equivalent to analyzing the graph to determine if 
the new window is already present in the map. 

Referring to claim 3, the many-to-many attribute of the "Opens Relation", as disclosed by 
Anodide et al., implies that the same window could be opened by more than one operation. 
Additionally, the "Opens Relation" is used to construct a graph representative of the GUI of an 
application, wherein the nodes of this graph represent windows of the GUI and edges of the 
graph represent operations that open the windows they point to. Thus, it is conceivable that two 
edges could point to a node in the graph, meaning that two operations could open the same 
window. This is equivalent to adding a shortcut to a window that is already present in the map. 

Regarding claim 6, it is shown above that in the method of Anodide et al., each new 
window is compared to windows that are referenced in the "Opens Relation", which is 
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equivalent to analyzing the graph to determine if the new window is already present in the map. 
Also as discussed above, the "Opens Relation" is a set of (op, win) pairs, where "win" is a 
reference to a window. The value for "win" can be the name of the window to which it refers. 
Thus, the comparison in the method of Anodide et al. implies a comparison of window names, 
where the new window is determined to already be present in the "Opens Relation" if the new 
window and the window in the "Opens Relation" have the same name. And as discussed above, 
since the "Opens Relation" is used to create the map, this is equivalent to determining that the 
new window is already present in the map if the new window and a window in the map have the 
same name. 

Referring to claim 11, reference letter B in FIG. 6 of the Anodide et al. application refers 
to a graph, representing the GUI of an application, on a computer. The graph is of a similar 
structure and functionality as a map and can therefore be considered equivalent. 

Referring to claim 12, the map shown in FIG. 6 of Anodide et al. is hierarchical, includes 
windows, and the edges of the map represent user actions. No user interface objects are shown 
in the map. However, the method described by Anodide et al. uses a "GUI object forest" to 
create the map. As described previously, the "GUI object forest" is a collection of trees whose 
roots represent windows of the GUI and whose nodes represent GUI objects in the windows. 
Thus it would be a trivial task to add GUI objects to the map of FIG. 6 by simply replacing each 
node in the map of FIG. 6 by its corresponding tree in the "GUI object forest." 
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In regards to claims 13 and 14, it was described above, in the discussion about claim 1, 
that the system of Anodide et al. requires that actions be performed on graphical user interface 
objects in order to perform a "tour of the interface". It is well known in the art that graphical 
user interface objects can include buttons, sliders, check boxes, and tab controls. Additionally, it 
is well known in the art that actions to be performed on user interface objects can include left 
mouse clicks, right mouse clicks, left mouse double clicks, and keystrokes. 

Referring to claim 15, the "computer program product" declared in claim 15 is inherent 
in the "computer implemented method" declared in claim 1. Therefore claim 15 is rejected for 
the same reasons that claim 1 is rejected. Additionally, the abstract of Anodide et al declares 
that the system of Anodide et al. utilizes "recorded test sequences and test designs". Recorded 
test sequences imply a computer storage medium. 



Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over Anodide et al. in 
view of Parker et al. Anodide et al. discloses a computer implemented method of mapping a 
graphical user interface that is equivalent to the method disclosed in claiml. However, Anodide 
et al. does not disclose that the method comprises receiving user input specifying that one or 
more graphical user interface objects should be ignored when generating the map. However, in 
column 29, line 5 of Parker et al. it is disclosed that when performing a comparison of two 
windows, a user can be allowed to mask out areas of a window. This implies that a user can 
specify that one or more graphical user interface objects can be ignored; the user can mask out 
areas of the window that the objects are located. Therefore, it would have been obvious to one of 
ordinary skill in the art, having the teachings of Anodide et al and Parker et al. before him at the 
time the invention was made, to modify the method of mapping a graphical user interface taught 
by Anodide et al. to include the method of comparison of Parker et al., where a user can specify 
that one or more graphical user interface objects should be ignored. It would have been 
advantageous to one of ordinary skill to utilize such combination because windows often contain 
areas where differences are considered insignificant and should be ignored, as taught by Parker 
et al. 

Claim 16 is rejected under 35 U.S.C. 103(a) as being unpatentable over Anodide et al. in 
view of the state of the art. As discussed above, Anodide et al. discloses a computer program 
product for mapping a user interface equivalent to that of claim 15. However, Anodide et al. 
does not acknowledge that the computer readable medium expressed in claim 1 5 is a CD-ROM, 
floppy disk, tape, flash memory, system memory, hard drive, or data signal embodied in a carrier 
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wave. Nevertheless it is extremely well known in the art that a computer readable medium could 
be a CD-ROM, floppy disk, tape, flash memory, hard drive, or data signal embodied in a carrier 
wave. The examiner takes OFFICIAL NOTICE of this teaching. Therefore, it would have been 
obvious to one of ordinary skill in the art, having the teachings of Anodide et al. before him, to 
modify the computer program product of Anodide et al. such that the computer readable medium 
could be a CD-ROM, floppy disk, tape, flash memory, system memory, hard drive, or data signal 
embodied in a carrier wave. It would have been advantageous to one of ordinary skill to utilize 
such combination because CD-ROMs, floppy disks, tapes, flash memories, hard drives, and data 
signals embodied in carrier waves are all conventional mediums for data storage and experience 
a lot of use, as made known by the state of the art. 

Claims 17 through 21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Anodide et al. in view of Mongan et al. Anodide et al. discloses a visual test design system that 
can programmatically execute an application to generate a map of the graphical user interface of 
the application, as is discussed above. The application mapper of Anodide et al. generates the 
map by performing a "tour of the interface", which is equivalent to recursively opening windows 
in the interface to identify new windows and add new windows to the map. As discussed above 
in the rejection for claim I, opening windows is equivalent to performing actions on the 
graphical user interface. In addition, the map of Anodide et al. is hierarchical and includes 
windows, graphical user interface objects and actions as is discussed above in the rejection for 
claim 12. Also as was shown earlier in the rejection for claim 13, the graphical user interface 
objects represented in the Anodide et al. map includes buttons, sliders, check boxes, and tab 
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controls. And as discussed earlier in the rejection for claiml4, the actions represented the map of 
Anodide et al. include left mouse clicks, right mouse clicks, left mouse double clicks, and 
keystrokes. Furthermore, Anodide et al. recognizes that a script generator can generate scripts 
that include instructions to test the application, and that these scripts can be executed by a 
capture/replay tool. 

However, Anodide et al. does not specifically disclose that the script generator utilizes 
the map to generate scripts. On the other hand, Mongan et al. discloses a test generator that 
iteratively selects paths through a cyclic directed graph in order to create scripts (see column 3, 
line 19 and column 5, linel3). This cyclic directed graph could be equivalent to a map of the 
graphical user interface; Anodide et al. acknowledges that a cyclic directed graph could represent 
the interface of an application. Thus, it would have been obvious to one of ordinary skill in the 
art, having the teachings of Anodide et al. and Mongan et al. before him at the time the invention 
was made, to modify the visual test design system taught by Anodide et al. to include the script 
generator of Mongan et al., because the script generator of Mongan et al. has a higher degree of 
automation than that of Anodide et al., making it less expensive. 



Allowable Subject Matter 

Claims 4, 5, 7, 8, and 10 are objected to as being dependent upon a rejected base claim, 
but would be allowable if rewritten in independent form including all of the limitations of the 
base claim and any intervening claims. The following is an examiner's statement of reasons for 
allowance: 
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In regards to claims 4 and 5, no prior was found that refers to a similarity threshold, 
wherein the similarity threshold is a percentage of graphical user interface objects that the new 
window and the window in the map have in common. 

Referring to claims 7, 8, and 10, no prior art was found that specifies user input 
concerning windows being considered the same or different, or how long to wait before 
identifying a second set of windows after performing an action. 

Any comments considered necessary by applicant must be submitted no later than the 
payment of the issue fee and, to avoid processing delays, should preferably accompany the issue 
fee. Such submissions should be clearly labeled "Comments on Statement of Reasons for 
Allowance." 



Conclusion 

The prior art made of record on form PTO-892 and not relied upon is considered pertinent to 
applicant's disclosure. Applicant is required under 37 C.F.R. § 1.1 1 1(c) to consider these 
references fully when responding to this action. The documents cited therein teach additional 
methods of mapping and analyzing graphical user interfaces. Additionally, the documents 
ftirther teach methods of utilizing graphical user interface maps to create scripts that include 
instructions to test applications. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Blaine T Basom whose telephone number is (703)305-7694. The 
examiner can normally be reached on 8:30 - 5:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703)308-3 1 16. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703)746-7238 for regular 
communications and (703)746-7240 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is 305-3900. 



btb 

July 22, 2002 
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